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An Evaluation of the Academic Volunteer and Mentor Service Program 



By David Illig, Ph.D. 



This report is in response to a legislative request that the California Research Bureau 
(CRB) evaluate the performance of the California Academic Volunteer and Mentor 
Service Act^ administered by the Governor’s Office of Child Development and Education 
(OCDE, now known as the Secretary of Education).^ Specifically, this report addresses 
an assessment of the impact of mentoring on student achievement, attendance, and 
behavior. 

BACKGROUND 

The goal of the Academic Volunteer and Mentor Service Program is to use volunteer 
mentors to provide support to children at risk of academic failure and to improve 
educational outcomes and behavior. The program was proposed by Governor Wilson 
during his First Inaugural Address as part of a set of proposals to “. . . shift from a remedial 
to a preventative government.” Chapter 901, Statutes of 1992, established the program; 
however, fianding did not become available until the 1996-97 fiscal year. During budget 
deliberations, program funding shifted from a General Fund appropriation to Proposition 
98 funding. As a result, administration of the program was absorbed within the OCDE 
budget. 

Literature Review 

“Academic Tutoring and Mentoring: A Literature Review,” summarizes the literature on 
mentoring. It is important to note that the quality of the studies cited in this report is, for 
the most part, poor. The one notable exception is the random assignment study of Big 
Brothers/Big Sisters. That program, however, is not an academic mentoring program. 

Assessment Strategy 

This report uses three data sources for the program assessment. Under the provisions of 
their grants, grantees must report annually specified data for each participant (treatment 
group) and similar data for a group of non-participants (control or comparison group). 
These data, which include participant and contrast group GPA, attendance, and behavior 
problem (such as expulsions, suspensions, or referrals) measures, provide the basic 
information for our assessment of program results. In addition, some grantees submitted 
evaluation reports from outside evaluators. Some of these reports contain statistical 
analyses of individual grantee projects and some contain survey data. Finally, grantees 
submit annual reports, which discuss various aspects of their program. Qualitative 
information from these reports was compiled and analyzed. 
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Statistical Analysis Strategy 

The analysis summarized here is limited to middle schools and high schools. The data for 
elementary schools was, by and large, of poor quality or too idiosyncratic for decent 
analysis. Further, the choice of analytical techniques is limited by the data (discussed 
below). Several data analysis strategies were examined. Among the options examined 
were discriminate analysis, regression analysis, analysis of variance, and t-tests. 

Combining data across grants also was explored. In addition, it was thought possible to 
use data from CDE data sets to provide school and grade level controls in order to deal 
with technical issues related to potential selection bias. Several data sets were constructed 
using CBEDS and STAR data; however, this approach was rejected on technical grounds. 

One basis for not pursuing this approach was that the data supplied by the grantees did not 
contain socioeconomic data on individuals in either the control or the treatment groups. 
Second, the GPA and attendance data could not be assumed to be comparable between 
schools or grant sites. Consequently, we chose to test each grant individually. 

We were left with three related techniques. Discriminate analysis was tested on data from 
a few schools and was found not to be superior to regression analysis. We rejected 
analysis of variance on the same grounds.^ Thus, we were left with two related 
techniques. Specifically, we chose to use t-scores and regression analysis. 

• T-scores to Test Differences Between Pre- and Post-GPA and Attendance. T- 
scores were run to determine whether statistically significant differences exist between 
the mean pre- and post-data for the treatment and control groups separately. Such a 
limited form of analysis provides almost no useful information about the value of 
mentoring. It does, however, allow for some preliminary indication about whether 
children in each group are improving, declining, or remaining the same across the 
school year. 

• T-scores to Test the Difference Between the Treatment and Control Group Data. 
T-scores were run to determine whether statistically significant differences exist 
between the mean change in pre- and post-data for the treatment group compared to 
the mean change in pre- and post-data for the comparison group. Using the t-score to 
test for the difference between the treatment and control group means allows the 
investigator to obtain some preliminary information about whether the mentoring 
program provides any benefit. Such information is limited because this form of 
analysis provides no control for differences between the two groups.^ 

• Regression Analysis to Test for Differences Between the Treatment and Control 
Groups. Regression analysis is used for all grants for which there was sufficient data. 
This technique provides the same result for schools with minimal data but also allows 
for inclusion of other data such as beginning GPA, grade level, sex, or beginning 
attendance when available. Including beginning GPA and other variables allows for 
some limited ability to control for differences between the treatment and control 
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groups. In that regard, the results produced by regression analysis may be somewhat 
more determinative than t-score results. 

The use of t-scores either to examine the difference between pre- and post-scores for each 
group or to examine the difference between the treatment group and contrast group means 
provides little usable information about the success of the mentoring programs. Thus, this 
report focuses on regression results run for each grant site separately, except when 
specifically discussed. 

Data Problems 

Several problems with the data supplied by grant site personnel for this analysis make any 
conclusions, at best, tentative. Included among the problems are the following: 

• Choice of Outcome Variables. The grantees were required to report GPA, 
attendance, and discipline (referrals, suspensions, and expulsions). Problems with each 
kind of outcome variable were found among the grantee data sheets. First, GPA data 
appears reliable only for middle schools and high schools and then only for each school 
or grantee. Comparability between schools may be limited because GPA data have a 
subjective component of unknown magnitude. Most elementary schools either didn’t 
collect GPA data or used some other kind of score that appeared idiosyncratic to the 
school such as running record scores. Second, attendance usually was reported as a 
ratio of absences to total days of school, but it isn’t clear whether absences included 
both excused and unexcused or only unexcused. Further, some schools reported 
number of days absent while others reported either nothing or something else. Third, 
many schools did not report disciplinary data on referrals and of those that did, it was 
not clear whether consistent definitions or consistent site-level policies were used. 
Suspension and expulsion data generally were not useful because too few suspensions 
or expulsions occurred to provide needed variability in the data. 

• Missing Data. The analysis also was made difficult because grantee data sheets were 
incomplete. Specifically, too frequently, either a beginning or ending GPA score was 
missing. In some cases both the beginning and ending scores were not reported. 
Similar problems occurred with the attendance and discipline data. The incomplete 
records could not, in most cases, be used in the data analysis.® In addition, a few 
grantees provided individual data for children who received mentoring services but 
only provided summary data for children in the comparison groups (e.g., Maclay 
Middle School). Thus, statistical analysis was not possible. Eliminating the 
incomplete records is the equivalent to sample attrition - some of which undoubtedly 
also occurred. Without socioeconomic data on children in the treatment and 
comparison groups, it is not possible to determine with any degree of certainty 
whether the attrition results in bias in the analytical results. 

• Assignment to Treatment or Contrast Group. In only one instance (Santa Barbara 
Elementary School District) did a grantee indicate the use of random assignment to 
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select children into either the treatment or control group. In a few instances, site 
personnel used four or five characteristics (gender, ethmcity, grade level, and 
socioeconomic status) to “match” comparison group children with treatment group 
children — such as Fresno USD and Huntington Beach UHSD. The remaining school 
sites used some other means for selecting children into the treatment or contrast 
groups. In those cases, the selection must be considered nonrandom. Thus, there is 
no assurance that, on average, each group will have the same preprogram 
characteristics. Further, in the absence of sociodemographic information for each 
child it is not possible to control for potential selection bias. 

In summary, the data provided by grantees is of questionable value for this evaluation. 
Consequently, the value of any conclusions derived from the quantitative analysis is 
limited. 

Results 

This section summarizes results obtained from the data analysis and an analysis of grantee 
annual reports. 

CRB Analysis of Outcomes Data 

The analysis reported in this section focuses on middle schools and high schools. Table 1 
summarizes the data analysis for those schools. Except for the simple t-score analysis, all 
analyses used the difference between the beginning GPA or attendance and the ending 
values of those variables. These differences, or so-called gain scores, prove superior for 
statistical analyses than analyses using either final GPA or final attendance data because 
gain scores allow for use of beginning GPA or beginmng attendance as control variables. 
This is important because there is so little data available to use in controlling for pre- 
program differences between the comparison and treatment groups. 

Simple T-scores. Simple t-tests for changes in GPA and attendance were performed. 
Numerous statistically significant results were found. Some of the statistically significant 
results suggest change in the “wrong” direction. For example, when it was expected that 
GPA would increase it was found that scores actually declined. Such poor results, in 
isolation, could suggest that the program didn’t work. One cannot put any weight on 
these results, however, because they do not consider any factors that might affect the 
results. In particular, they do not consider that the control group may also have 
experienced declining scores but may have suffered greater declines than the treatment 
group. Also, simple t-scores do not allow for consideration of sociodemographic 
differences between groups that also could affect the results. 

T-scores: Difference Between Group Means. Computing t-scores on the difference 
between the mean score for the treatment group and the mean score for the contrast group 
-the gain from pre- to post-assessments - allows one to examine whether one group 
progressed more than the other group. For example, assume a grantee reports that its 
treatment group exhibited an average increase in GPA between the beginmng of the year 



California Research Bureau, California State Library 



4 



and the end of the year of .5 while the contrast group experienced an increase in GPA of 
.2. One can use a t-score to determine whether the difference between the treatment 
group mean (.5) and the contrast group mean (.2) is statistically significant. Table 1 
reports t-score results for those high schools and middle schools analyzed. Table 2 
provides a summaiy of the statistically significant results from Table 1. In no instance did 
more than about 28 percent of sites produce significant results. Of those sites, several 
show attendance results that have the wrong sign. 

Regression Results. Regression analysis can produce more useful results when compared 
to t-scores since this statistical tool allows for the use of additional data, when available, 
for use in controlling for potential bias. As discussed above, there were few additional 
data elements available for this purpose, however, we were able to incorporate beginning 
GPA scores and, where available, data on gender, grade level, beginning attendance, and 
beginning referrals in order to control for some differences in the groups. The first part of 
Table 1 reports summary information on the statistical significance of the “group” variable 
(a 0,1 variable that identifies whether the child was in the treatment [1] or comparison [0] 
group). 

Regression equations were run for high schools and middle schools and for GPA and 
attendance. Table 2 summarizes the statistically significant results from Table 1. As with 
the t-score results there were relatively few sites that have statistically significant results 
for the group variable. In three instances, we found statistically significant results for the 
attendance outcome but with the wrong sign. In these instances the wrong sign means 
that the comparison group reported larger attendance gains than did the treatment group. 

Information in Grantee Annual Reports 

In addition to the data submitted by grant sites, grant sites also submitted armual reports 
on their projects. These reports were analyzed and several findings emerged, including: 

• Several grantees indicated problems bringing mentors into their programs due to the 
new fingerprinting requirements. In particular, the time needed to complete the 
fingerprint checks caused mentors to loose interest and drift away. These problems do 
not appear to affect all sites equally and may only be a transient problem. Further 
examination of this issue may be warranted if the delays continue. 

• Some schools report difficulty finding, retaining or otherwise keeping mentors 
engaged in their programs. Further, some sites report a lack of commitment, poor log 
keeping, and erratic schedule keeping by mentors, (e.g., Los Angeles COE, San Juan 
USD, and Fresno USD) 

• A few sites indicated that they experienced problems coordinating with or receiving 
buy-in from teachers and/or principals. Again, this appears to be a problem at only a 
few sites but examination of how site coordination and planning occur may be 
warranted, (e.g., Ravenswood City ESD) 
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• Many sites surveyed some combination of mentors, mentees, site personnel and 
parents. Generally, the results are positive, particularly improvements in engagement, 
self-esteem, and desire to remain in the mentoring program. Little generalization of 
these results is possible since the surveys varied from one school to the next in design, 
manner in which data were collected, and in the individuals surveyed. At least one 
school reported that mentors felt mentees were not motivated and that little parent 
involvement existed. Two sites reported changes in behavior or attitude that became 
more negative following mentoring. 

• Sites engaged in a variety of mentoring approaches. For example, some sites used 
peer mentors, some used adult mentors - either from the community or from school 
personnel, still others used cross-age mentors. At least one site used multiple types of 
mentoring (Eureka HSD). A variety of settings also were used. For example, Oroville 
Union HSD used businesses as the mentoring setting. Others allowed some flexibility 
as to where the mentors met. Some sites used either a room at the school site or the 
school library for mentoring sessions. In addition, there was some variation in the 
time of day when mentoring occurred. Some sites used time during the school day 
(e.g.. Standard ESD), others used after school times (e.g.. Pleasant Valley SD and 
Redwood City ESD), and at least one site used lunch time (San Francisco USD). 

• Sites varied in the number of hours per week that mentors were expected to spend 
with mentees. Most sites emphasized one-on-one mentoring; however, some sites 
used group sessions (Los Angeles USD - El Camino Real HS). 

• Some sites used parent contracts (e.g.. Elk Grove USD) while others used mentor- 
mentee contracts (e.g.. East Side Union HSD). Other sites, such as California 
Academy of Mathematics and Science, found mentors from among the math and 
science community; or San Mateo County Office of Education found Latina 
professionals for its Hispanic girl mentees. 

• Some sites developed action planning processes so mentees and mentors could identify 
issues and set goals (e.g., Fresno USD and Mount Diablo USD). Other sites, such as 
Oroville Union HSD and Fremont Union HSD, developed supplemental classes, 
sessions or seminars to support the mentoring sessions or obtain greater parent 
involvement. 

Schools with Statistically Significant Results 

This section examines sites reporting statistically significant results for GPA, attendance or 
referrals in order to determine whether those sites have different characteristics than other 
sites. This analysis begins with the regression analysis and abstracts information from 
qualitative and survey information provided by sites either in their annual reports or from 
outside evaluators. Tables 1 and 2 summarize the statistically significant regression 
results. Table 3 summarizes the compilation of qualitative information for these sites. 
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Several middle school and high school sites show significant results for both GPA and 
attendance. Care must be taken, however, when interpreting these results. For example, 
site number 9616 reported both highly statistically and academically significant GPA 
results. Specifically, the treatment group exhibits about 0.7 of a point increase in GPA 
when compared to the comparison group. Notwithstanding this seemingly great outcome, 
both the treatment and comparison group GPAs declined between the pre-and post- 
assessments - but the GPAs for the comparison group declined to a greater degree. 
Alternatively, at sites 9624 and 9621 the significant results came from declining 
comparison group GPAs and small increases in the treatment group GPAs. While such 
results suggest that the program stemmed a potential decline in GPA for children in the 
treatment group, it is possible that the comparison group contains individuals with 
different characteristics than the individuals in the treatment group. 

Finally, two high school sites and one middle school site reported statistically significant 
differences in the attendance variable but which show that the comparison group had 
superior attendance gains relative to the treatment group. For example, site 9662 reported 
a decline in attendance for both groups but the treatment group exhibited a greater decline 
in attendance than did the comparison group. Another example is the middle school at site 
9609, where both groups reported attendance declines; however, the treatment group 
experienced a 15 percentage point decline relative to the comparison group. 
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CONCLUSIONS AND RECOMMENDATIONS 
Conclusions 

Drawing so-called best practices from the annual reports of “successful” sites is difficult. 
Examination of the annual reports and evaluation reports provides little information that 
could be used to distinguish the successful from the not-so-successful sites. Some 
practices that might warrant further investigation include: 

• Linking mentoring to career or job development - at least, for high school programs. 

• Individualized plans identifying needs and goals to direct the mentoring program. 

• Use of contracts between mentors and mentees - and, perhaps, parents. 

• Use of incentives and recognition to celebrate progress. 

The next phase of evaluation should include a specific component focused on issues 
related to site operations in order to develop best practices and guidelines. 

Also, some sites produced statistically significant results for some outcome variables. 
Because the data used in these analyses is of limited quality, it is not possible to say that 
these results provide either general or specific indications that the Academic Mentoring 
and Tutoring programs studied in this paper are successful. Nonetheless, these results 
offer support for the idea that academic mentoring may provide benefits for some children. 
This weak endorsement should be viewed with caution. There is much work yet to 
accomplish in order to draw any conclusions - either positive or negative. 

Recommendations 

Our experience with this assessment leads to several recommendations for consideration in 
future rounds of grants. 

• Standardize and Enforce Data Collection Requirements. The first set of grants 
included a requirement to collect and report certain data on children receiving 
mentoring services and data for some comparison group. While it was appropriate to 
focus data collection on outcomes such as those highlighted in SB 1 1 14 - GPA, 
attendance, referrals, and suspensions - insufficient effort went into ensuring that all 
sites used standardized definitions of the variables of interest. Further, little effort 
went into enforcing complete reporting of the required data. The latter issue appears 
to be the result of a lack of funding for program staff. Grant sites also were not 
required to provide information about the duration, intensity or length of time that 
each child was in a mentoring program. In addition, the grants did not require sites to 
report sociodemographic information for children in the treatment or comparison 
groups. Finally, while some schools provided achievement data from norm referenced 
tests, there was no requirement either that a specific test or that a specific form of test 
results be reported. 
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Consultation with program staff resulted in stronger reporting requirements for the 
second round of grants awarded in the fall of 1998. Included in these enhanced 
reporting requirements is a requirement that sites provide better information about 
how their comparison group was chosen and about the type of mentoring program 
they offer. Further, sites should report STAR test results for each child in both groups 
and should provide the sociodemographic data collected by the STAR exams. Finally, 
sites should provide information on duration (program entry and exit dates), number of 
mentoring sessions, and length of sessions. Program staff also should require sites to 
include site codes (so-called CDS codes) for individual children in both the treatment 
and comparison groups. Requiring information about the school each child attends 
would allow evaluators to match individual data from the mentoring program to 
school site and grade level data contained in databases maintained by the California 
Department of Education (specifically, the CBEDS and STAR data sets). 
Notwithstanding these improved reporting requirements, useful and complete 
analytical data will require that program staff enforce better and more consistent 
reporting than has been the case to the present. 

• Consider Funding Longer Term FoIIow-Up Analyses of Programs at Some Sites. 
Current reporting requirements only require the reporting of pre- and post-treatment 
outcome data and other data. In order to determine whether academic mentoring 
provides benefits over a longer period than the year of the program, it would be useful 
if program staff would consider funding some longer-term follow-up data collection. 
Such data collection should be limited to sites with stronger research designs or to 
sites that would agree in advance to provide ongoing information about children who 
participated in their treatment and comparison groups. 

• Develop Assessments of Alternative Mentoring Approaches to Determine 
Efficacy of Each Type of Mentoring. Sites used a variety of approaches to 
mentoring including peer, cross-age, and adult mentors. Some sites also used group 
mentoring as well as one-on-one mentoring. Finally, some sites used a variety of 
settings for delivery of mentoring services. Stronger data collection on these differing 
mentoring designs could provide data that allow subsequent assessments to determine 
which of these alternatives is more effective. 

• Develop Guidelines Based on Best Practices to Support New Grantees. Several 
sites included pleas in their annual reports expressing a desire for the OCDE staff to 
produce guidelines so that they would have better criteria and support during their 
program development process. It is possible that OCDE staff wanted “100 flowers to 
bloom” so better information about alternatives could inform future rounds of grants. 

It is also possible that limitations on staffing prevented early development of 
guidelines. Finally, it is possible that staff determined that guidelines would not be 
possible for the first round of grants because little was known about what works. 
Nevertheless, increased interest in mentoring has stimulated interest in developing 
practice guidelines. Program staff might benefit from a survey of that literature and 
from some time spent with a task force that includes program staff, grantees, and 
researchers to develop guidelines and other mechanisms to support grant sites. 
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• Develop Instruments to Measure Outcomes Such as Attitude and Behavior 
Change. SB 1 1 14 emphasized outcomes such as academic achievement, attendance 
and discipline as the basis for determining success or failure of the Academic 
Mentoring program. As indicated above, GPA may be the wrong achievement measure 
given that STAR test results are now available. Further, while discipline data such as 
referrals, suspensions, and expulsions may be appropriate measures of disruptive 
behaviors, they may not capture the behavioral changes most likely to occur from 
mentoring. The Secretary may wish to consider testing or developing other tools to 
measure altitudinal or behavioral change. 




California Research Bureau, California State Library 



10 



APPENDIX 



TABLE 1: Summary of Regression and T-score Data 

TABLE 2; Number of Schools for Which Statistically Significant Results Were Found 
TABLE 3: School Sites with Statistically Significant GPA or Attendance Variables 



O 

ERIC 



California Research Bureau, California State Library 



11 



Table 1: Summary of Regression and T-score Data 

(all significance tests are two-tail tests except where noted) 
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Table 2: Number of Schools For Which Statistically Significant Results Were 
Found 

(significance defined as the 5% level for a two-tailed t-test) 




Regression 




T-Scores 




GPA 


Attendance 




GPA 


Attendance 


High Schools 














Number 

significant 


5 


4(+),2(-) 




5 


2(+) K-) 














Total tests 


21 


19 




21 


18 














Middle 


Schools 














Number 

significant 


3 


1(+), l(-) 




2 


2(+) l(-) 














Total tests 


18 


16 




15 


11 
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Table 3: School Sites with Statistically Significant GPA or Attendance Variables 





High School Sites 


East Side UHSD, Santa 
Teresa High 


• Many students have mentoring contracts signed by student, parent, 
and mentor. 

• Tutoring occurs during and after school. 

• Used activities to reward mentors and mentees and to advertise 
program. 

• Difficulties increasing mentor numbers because loss of current 
mentors only replaced by new mentors. 

• Mentee surveys indicate that mentees believe that they have 
improved in their classes. 

• Surveys also indicate that mentees have more confidence in their 
personal and academic ability. 

• Mentees give mentors high ratings. 


Fremont UHSD, Fremont 
High 


• Program uses both adult and peer tutors. 

• Established a tutoring center for students. 

• Tutors and students make a one-semester commitment. 

• Held annual dinner to recognize efforts of mentors and mentees. 

• Added “life skills” component to tutorial sessions. 


Los Angeles USD, El 
Camino Real High 


• Two types of mentoring; one-on-one, which actually included both 
an adult and a peer tutor and group tutoring that used groups of 10 
to 12 students. 

• Activities such as trip to Broadway show for students that make 
improvements. 

• Discuss anger management, responsibility and goal setting during 
group sessions. 

• Mentees offered planning sessions with college counselor. 

• Received donations from local businesses to help support program. 


Oroville UHSD, Las Plumas 
High 


• Mentoring occurs at local businesses. 

• Mentors encouraged to offer educational guidance, job seeking 
strategies, communication skills, and self-management skills. 

• Mentees receive training on a variety of interview, time 
management, and communication skills, and receive guidance on 
need to improve academic performance before placement with 
business people. 

• Mentees can be referred for additional training by teachers. 

• Many mentees have received job opportunities. 

• Annual report highlights goal setting, direction and guidance as key 
elements in forming link between business community and schools. 


Sacramento COE, 
Alternative Ed. 


• Mentoring sessions held on campus at fixed times. 

• Estimate average mentoring time at about 2 hours per week. 

• Three mentoring cycles per year and about 40 percent attended 
more than one cycle. 

• Many mentees have difficult problems, are wards of the court, or are 
formerly incarcerated. 

• Transient population makes intervention difficult. 
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San Diequito UHSD, San 
Diequito High 


• Uses portfolios to mark student progress toward goals. 

• Assemblies for mentees include mentor descriptions of their jobs 
and careers. 

• Job shadowing a primary focus for mentors. 

• Mentees seem to appreciate their experiences with their mentors. 


San Mateo UHSD, 
Peninsula High 


• Primary focus on career training and job site mentoring. 

• Program featured in front page article in local paper. 

• Indicate that truancy is a problem. 

• Recruitment used PSAs on TV, mailers, and meetings at job sites. 


Merced UHSD, Livingston 
High 


• Positive article in local paper and influenced elementary principal 
to become a mentor. 

• Use a community coordinator for recruitment of mentors. 

• Mentors attend support sessions and field trips for mentees. 

• Most of the mentoring activity appears to be of the Big Brothers/Big 
Sisters variety - not academic mentoring. 

• Mentees put on a dinner for mentors at end of school year. 


Middle School Sites 


Berkeley USD, Willard 
Middle 


• Set goals for mentees to increase grade by one letter for some 
subject. 

• Goal was 4 hours per week of academic mentoring - report suggests 
the program met this goal. 

• Mentor-mentee activities included a talent show fundraiser, trips to 
Great America, college campus visits. 


Los Angeles USD, Foshay 
Learning Center 


• Set overall goal of bringing all tutored students to grade level in 
reading. 

• Uses individual long range plans to guide mentor-mentee efforts. 

• Uses structured academic mentoring experience that uses at least 
1.5 hours per week. 

• Mentors and mentees follow individually set schedules. 

• Many mentors from USC who do mentoring as part of class. 

• Of the 8^^ to 10* grade mentees that read at 3*^^ grade level or below, 
all are now at grade level. 


Ravenswood City ESD, 
49er Academy 


• Anticipates mentor-mentee relationship will go beyond academic 
tutoring to “positive adult relationship.” 

• Estimate average mentoring time about 2.5 hours per week. 

• Females received one group tutoring and one one-on-one tutoring 
session per week. 

• Sets stringent standards for all mentors and mentees to ensure 
quality of program. 

• Encourages job shadowing. 

• Supported group activities such as BBQs and ice skating trip. 

• Integrated a community service component into program. 

• Problems with principal and teacher buy-in. 

• Consistent lack of parent involvement. 

• Teacher surveys indicate that mentees have an improved sense of 
self-worth, better communication skills, and self-discipline and 
punctuality. 

• Mentee surveys indicate that mentors were influential in building 
self-worth and that mentees expressed interest in mentor’s 
occupations. 

• Local COSTCO assisted in providing and training mentors. 
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Rowland USD, Giano 
Intermediate 


• Used some kind of matching process to identify comparison group, 

• Evaluator conunends recruitment methods and team building 
efforts, 

• Mentees surveyed indicated desire to see mentors more often, 

• High proportion of mentees indicates that mentors helped with 
discipline problems and feelings about self, 

• Mentors recruited using both Spanish and English language media 
such as the Pennysaver and Spanish language TV, 


Coronado USD, Coronado 
Middle 


• Use seniors for peer tutoring and both military and local service 
organizations, 

• Use part time coordinator who has no office due to lack of space, 

• Limited access to data due to lack of computer access. 


Emery USD, Emery Middle 


• Use a curriculum developed by Consortium on Reading Excellence 
- a reading program — to support mentors; however, the focus of 
the mentoring program was on math improvement, 

• Mentors meet for one hour per week on campus, 

• Interviews with mentors and with students suggest some mismatch 
between groups regarding preferred times for mentoring sessions, 

• Have mentors from private industry, city parks department, and 
high school cross-age mentor/tutors, 

• Use PSAs, e-mail, community TV, and announcements at school 
sites and district offices to seek mentors and advertise their 
program, 

• Use district and school site recognition ceremonies to honor 
mentors. 
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ENDNOTES 



^ Chapter 901, statutes of 1992 (SB 1114, Leonard). 

^ Item 0558-001-0001, Supplemental Report to the 1998 Budget Act. 

^ Since discriminate analysis, analysis of variance, and regression analysis are derived from the General 
Linear Model and can be made to provide similar results. Consequently, we chose to use the analytical 
form with which we were most familiar and seemed most accessible for the data. 

^ For very simple cases — where regression analysis does not include conditioning variables, regression 
analysis produces results that are identical to t-scores. 

^ In the absence of random assignment between a treatment and control group, differences between the 
populations must be assumed. Such differences can produce differences in measured outcomes for reasons 
that have nothing to do with the intervention - in this case some form of mentoring. Important 
measurable differences between the treatment and control groups that could affect outcomes include, but 
are not limited to, maternal educational attainment, limited English proficiency, poverty status, or 
previous or current learning problems. In addition, many difficult-to-measure or unmeasurable 
characteristics could differentially influence treatment and control group outcomes and bias any results. 

^ For example, at one point we constructed a combined middle school database from 14 middle school 
sites. Those 14 schools included 1985 treatment and comparison group records. When records with 
missing data were removed, we were left with 1013 records for our regression analysis. This is a loss of 
about 49 percent of the records. There is no way to determine whetlier the remaining records for the 
treatment and comparison groups represent similar populations (assuming this was possible - wliich it 
isn’t - for the universe of records). 
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